Steganographic techniques for securely delivering electronic digital rights management control information over insecure communication channels

ABSTRACT

Electronic steganographic techniques can be used to encode a rights management control signal onto an information signal carried over an insecure communications channel. Steganographic techniques ensure that the digital control information is substantially invisibly and substantially indelibly carried by the information signal. These techniques can provide end-to-end rights management protection of an information signal irrespective of transformations between analog and digital. An electronic appliance can recover the control information and use it for electronic rights management to provide compatibility with a Virtual Distribution Environment. In one example, the system encodes low data rate pointers within high bandwidth time periods of the content signal to improve overall control information read/seek times.

CROSS REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. application Ser. No.10/189,231, filed Jul. 5, 2002 now U.S. Pat. No. 6,618,484, which is acontinuation of U.S. application Ser. No. 09/790,566, filed Feb. 23,2001, now U.S. Pat. No. 6,449,367, which is a continuation of U.S.application Ser. No. 09/247,328, filed Feb. 10, 1999, now U.S. Pat. No.6,240,185, which is a continuation of U.S. application Ser. No.08/689,606, filed Aug. 12, 1996, now U.S. Pat. No. 5,943,422.

This application is related to commonly assigned application Ser. No.08/388,107 of Ginter et al., filed 13 Feb. 1995, entitled “SYSTEMS ANDMETHODS FOR SECURE TRANSACTION MANAGEMENT AND ELECTRONIC RIGHTSPROTECTION, now abandoned. A file wrapper continuation of applicationSer. No. 08/388,107 issued as U.S. Pat. No. 5,982,891. We incorporate byreference, into this application, the entire disclosure of thisprior-filed Ginter et al. patent application just as if its entirewritten specification and drawings were expressly set forth in thisapplication.

FIELD OF THE INVENTION

The present inventions relate generally to computer security, and moreparticularly to steganographic techniques for hiding or encodingelectronic control information within an information signal carried byan insecure communications channel. Still more particularly, the presentinventions relate to systems, methods and techniques that substantiallyinvisibly and/or indelibly convey, over analog or other insecurecommunications channels, digital rights management control informationfor use within a virtual distribution environment electronic rightsmanagement system.

BACKGROUND AND SUMMARY OF THE INVENTION

The world is becoming digital. Digital signals are everywhere—in ourcomputers, television sets, VCRs, home stereos, and CD players. Digitalprocessing—which operates on information “bits” (numerical “on” or “off”values)—provides a degree of precision and protection from noise thatcannot be matched by the older, “analog” formats we have used since thebeginning of the electronic age.

Despite the clear advantage of digital communications, the older“analog” domain remains significant. Many of our most importantinformation delivery mechanisms continue to be based on analog—notdigital—signaling. In fact, most of our electronic entertainment, news,sports and music program material comes to us in the form of analogsignals. For example:

-   -   Television remains largely analog. Although the distribution of        television programming to local cable systems is increasingly        digital and most modern television sets include digital signal        processing circuits, the local cable television “head end”        continues to send television signals to the subscriber's set top        box and television in analog—not digital—form. It will cost a        great deal to convert local cable distribution from analog to        digital. In the United States, for example, the widespread        conversion from analog to digital television is projected to        take no less than 15 years and perhaps even longer.    -   In radio broadcasting, too, analog communication continues to        reign supreme. Thousands of radio stations broadcast music, news        and other programs every day in analog form. Except for a few        experimental digital systems, practically all radio broadcasting        is carried over analog communications channels.    -   The movies and videos we rent at the local video tape rental        store are analog.    -   Commercially available music tape cassettes are recorded in        analog formats.

Moreover, the “real world” is analog. Everything digital must ultimatelybe turned into something analog if we are to experience it; andconversely, everything analog must be turned into something digital ifthe power of modern digital technology will be used to handle it. Moderndigital technology also allows people to get better quality for lessmoney.

Despite the pervasiveness of analog signals, existing methods formanaging rights and protecting copyright in the analog realm areprimitive or non-existent. For example:

-   -   Quality degradation inherent in multigenerational analog copying        has not prevented a multi-billion dollar pirating industry from        flourishing.    -   Some methods for video tape copy and pay per view protection        attempt to prevent any copying at all of commercially released        content, or allow only one generation of copying. These methods        can generally be easily circumvented.    -   Not all existing devices respond appropriately to copy        protection signals.    -   Existing schemes are limited for example to “copy/no copy”        controls.    -   Copy protection for sound recordings has not been commercially        implemented.

A related problem relates to the conversion of information between theanalog and digital domains. Even if information is effectively protectedand controlled initially using strong digital rights managementtechniques, an analog copy of the same information may no longer besecurely protected.

For example, it is generally possible for someone to make an analogrecording of program material initially delivered in digital form. Someanalog recordings based on digital originals are of quite good quality.For example, a Digital Versatile Disk (“DVD”) player may convert a moviefrom digital to analog format and provide the analog signal to a highquality analog home VCR. The home VCR records the analog signal. Aconsumer now has a high quality analog copy of the original digitalproperty. A person could re-record the analog signal on a DVD-R (aDigital Versatile Disk appliance and media supporting both read andwrite operations). This recording will in many circumstances havesubstantial quality—and would no longer be subject to “pay per view” orother digital rights management controls associated with the digitalform of the same content.

Since analog formats will be with us for a long time to come,rightsholders such as film studios, video rental and distributioncompanies, music studios and distributors, and other value chainparticipants would very much like to have significantly better rightsmanagement capabilities for analog film, video, sound recordings andother content. Solving this problem generally requires a way to securelyassociate rights management information with the content beingprotected.

People have for many years been-using various techniques allowingdigital information to, in effect, ride “piggyback” on analoginformation signals. For example, since the 1960s, it has been common todigitally encode text information such as subtitles into otherwiseunused portions of analog television signals (e.g., within the so-called“Vertical Blanking Interval”).

Unfortunately, sending digital information using such known digitalencoding techniques is problematic because the digital information isnot persistent. It is relatively easy to strip out or eliminate digitalinformation encoded using prior techniques commonly employed forsuperimposing digital signals onto an analog information signal. Analogcommunications channels may commonly be subjected to various signalprocessing that may (intentionally or unintentionally) strip out digitalinformation added to the analog signal—defeating any downstream system,process or technique that depends on the presence and readability of thedigital information. For example, the television vertical blankingsignal—along with any signal components disposed within the verticalblanking interval—is typically routinely eliminated whenever a videosignal is processed by a computer.

Attempting to use insecure techniques for providing rights management isat best ineffective, and can be worse than no rights management at all.Unscrupulous people can strip out insecure control informationaltogether so that the corresponding information signal is subject to nocontrols at all—for example, defeating copy protection mechanisms andallowing users to avoid paying for rights usage. More nefariously, anunscrupulous person could alter an insecure system by substituting falsecontrol information in place of the proper information. Suchsubstitutions could, for example, divert payments to someone other thanlegitimate rights holders—facilitating electronic fraud and theft.

Prior, insecure techniques fail to solve the overall problem of how toprovide and securely manage advanced automatic electronic rightsmanagement for analog and other information signals conveyed over aninsecure communications channel. The lack of strong rights managementfor analog signals creates a huge gap in any comprehensive electronicrights management strategy, and makes it possible for consumers andothers to circumvent—to at least some extent—even the strongest digitalrights management technologies. Consequently, there is a real need toseamlessly integrate analog delivery models with modern electronicdigital rights management techniques.

The present inventions solve these and other problems by providing “endto end” secure rights management protection allowing content providersand rights holders to be sure their content will be adequatelyprotected—irrespective of the types of devices, signaling formats andnature of signal processing within the content distribution chain. This“end to end” protection also allows authorized analog appliances to beeasily, seamlessly and cost-effectively integrated into a modern digitalrights management architecture.

The present inventions may provide a Virtual Distribution Environment(“VDE”) in which electronic rights management control information may bedelivered over insecure (e.g., analog) communications channels. ThisVirtual Distribution Environment is highly flexible and convenient,accommodating existing and new business models while also providing anunprecedented degree of flexibility in facilitating ad hoc creation ofnew arrangements and relationships between electronic commerce and valuechain participants—regardless of whether content is distributed indigital and/or analog formats.

The present inventions additionally provide the following important andadvantageous features:

-   -   An indelible and invisible, secure technique for providing        rights management information.    -   An indelible method of associating electronic commerce and/or        rights management controls with analog content such as film,        video, and sound recordings.    -   Persistent association of the commerce and/or rights management        controls with content from one end of a, distribution system to        the other—regardless of the number and types of transformations        between signaling formats (for example, analog to digital, and        digital to analog).    -   The ability to specify “no copy/one copy/many copies” rights        management rules, and also more complex rights and transaction        pricing models (such as, for example, “pay per view” and        others).    -   The ability to fully and seamlessly integrate with        comprehensive, general electronic rights management solutions        (such as those disclosed in the Ginter et al. patent        specification referenced above).    -   Secure control information delivery in conjunction with        authorized analog and other non-digital and/or non-secure        information signal delivery mechanisms.    -   The ability to provide more complex and/or more flexible        commerce and/or rights management rules as content moves from        the analog to the digital realm and back.    -   The flexible ability to communicate commerce and/or rights        management rules implementing new, updated, or additional        business models to authorized analog and/or digital devices.

Briefly, the present inventions use “steganography” to substantiallyindelibly and substantially invisibly encode rights management and/orelectronic commerce rules and controls within an information signal suchas, for example, an analog signal or a digitized (for example, sampled)version of an analog signal.

The Greek term “steganography” refers to various “hidden writing” secretcommunication techniques that allow important messages to be securelycarried over insecure communications channels. Here are some examples ofsteganography:

-   -   In ancient Persia an important message was once tattooed on a        trusted messenger's shaved scalp. The messenger then allowed his        hair to grow back—completely hiding the message. Once the        messenger made his way to his destination, he shaved his hair        off again—exposing the secret message so the recipient could        read it on the messenger's shaved scalp. See Kahn, David, The        Codebreakers page 81 et seq. and page 513 et seq. (Macmillan        1967). This unusual technique for hiding a message is one        illustration of “steganography.”    -   Another “steganographic” technique encodes a secret message        within another, routine message. For example, the message “Hey        Elmer, Lisa Parked My Edsel” encodes the secret message “HELP        ME”—the first letter of each word of the message forming the        letters of the secret message (“Hey Elmer, Lisa Parked My        Edsel”). Variations on this technique can provide additional        security, but the basic concept is the same—finding a way to        hide a secret message within information that can or will be        sent over an insecure channel.    -   Invisible ink is another commonly used “steganography”        technique. The secret message is written using a special        disappearing or invisible ink. The message can be written on a        blank piece of paper, or more commonly, on the back or front of        the piece of paper carrying a routine-looking or legitimate        letter or other written communication. The recipient performs a        special process on the received document (e.g., exposing it to a        chemical or other process that makes the invisible ink visible)        so that he or she can read the message. Anyone intercepting the        paper will be unable to detect the secret message—or even know        that it is there—unless the interceptor knows to look for the        invisible message and also knows how to treat the paper to make        the invisible ink visible

The present inventions use steganography to ensure that encoded controlinformation is both substantially invisible and substantially indelibleas it passes over an insecure communications channel. At the receivingend, a secure, trusted component (such as a protected processingenvironment described in Ginter et al.) recovers thesteganographically-encoded control information, and uses the recoveredinformation to perform electronic rights management (for example, onanalog or other information signals carried over the same channel).

One specific aspect provided by the present inventions involvesteganographically encoding digital rights management controlinformation onto an information signal such as, for example, an analogor digitized television, video or radio signal. The steganographicencoding process substantially inextricably intertwines the digitalcontrol information with images, sounds and/or other content theinformation signal carries—but preferably without noticeably degradingor otherwise affecting those images, sounds and/or other content. It maybe difficult to detect (even with educated signal processing techniques)that the analog signal has been steganographically encoded with a rightsmanagement control signal, and it may be difficult to eliminate thesteganographically encoded control signal without destroying ordegrading the other information or content the signal carries.

The present inventions also provide a secure, trusted protectedprocessing environment to recover the steganographically-encoded controlsignal from the information signal, and to enforce rights managementprocesses based on the recovered steganographically encoded controlsignal. This allows the information signal delivery mechanism to befully integrated (and made compatible) with a digital virtualdistribution environment and/or other electronic rights managementsystem.

In accordance with yet another aspect provided by this invention,steganographically encoded, digital rights management controlinformation may be used in conjunction with a scrambled and/or encryptedinformation signal. The scrambling and/or encryption can be used toenforce the rights management provided in accordance with thesteganographically encoded rights management control information. Forexample, the control signal can be steganographically decoded and usedto control, at least in part, under what circumstances and/or how theinformation signal is to be descrambled and/or decrypted.

In accordance with yet another feature provided by the invention,digital certificates can be used to securely enforce steganographicallyencoded rights management control information.

In accordance with still another feature provided by the invention,steganography is used to encode an information signal with rightsmanagement control information in the form of one or more protectedorganizational structures having association with electronic controls.The electronic controls may, for example, define permitted and/orrequired operation(s) on content, and consequences of performing and/orfailing to perform such operations. The organizational structure(s) mayidentify, implicitly or explicitly, the content the electronic controlsapply to. The organizational structure(s) may also define the extent ofthe content, and semantics of the content.

The type, amount and characteristics of the steganographically encodedrights management control information are flexible andprogrammable—providing a rich, diverse mechanism for accommodating awide variety of rights management schemes. The control information canbe used to securely enforce straightforward secure rights managementconsequences such as “copy/no copy/one copy” type controls—but are by nomeans limited to such models. To the contrary, the present invention canbe used to enable and enforce much richer, more complex rightsmanagement models—including for example those involving usage auditing,automatic electronic payment, and the use of additional electronicnetwork connections. Moreover, the rights management controlarrangements provided by the present invention are infinitely extensibleand scaleable—fully accommodating future models as they are commerciallydeployed while preserving full compatibility with different (andpossibly more limited) rights management models deployed during earlierstages.

The organizational structure(s) may be steganographically encoded insuch a way that they are protected for purposes of secrecy and/orintegrity. The employed steganographic techniques may provide somedegree of secrecy protection—or other security techniques (e.g., digitalencryption, digital seals, etc.) may be used to provide a desired orrequisite degree of security and/or integrity protection for thesteganographically encoded information.

In one example, the organizational structure(s) may comprise digitalelectronic containers that securely contain corresponding digitalelectronic control information. Such containers may, for example, usecryptographic techniques. In other examples, the organizationalstructure(s) may define associations with other electronic controlinformation. The other electronic control information may be deliveredindependently over the same or different communications path used todeliver the organizational structure(s).

In one example, the steganographic techniques employed may involveapplying the organizational structure information in the form of highfrequency “noise” to an analog information signal. Spectral transformsmay be used to apply and recover such steganographically-encoded highfrequency “noise.” Since the high frequency noise components of theinformation signal may be essentially random, adding a pseudo-randomsteganographically encoded control signal component may introducesubstantially no discernible information signal degradation, and may bedifficult to strip out once introduced (at least without additionalknowledge of how the signal was incorporated, which may include a sharedsecret).

In accordance with another aspect provided by the invention, asteganographic encoding process analyzes an information signal todetermine how much excess bandwidth is available for steganographicencoding. The steganographic encoding process may use variable data rateencoding to apply more control information to parts of an informationsignal that use much less than all of the available communicationschannel bandwidth, and to apply less control information to parts of aninformation signal that use nearly all of the available communicationschannel bandwidth.

In accordance with still another aspect provided by the invention,multiple organizational structures may be steganographically encodedwithin a given information signal. The multiple organizationalstructures may apply to different corresponding portions of theinformation signal, and/or the multiple organizational structures may berepetitions or copies of one another to ensure that an electronicappliance has “late entry” and/or error correcting capability and/or canrapidly locate a pertinent organizational structure(s) starting from anyarbitrary portion of the information signal stream.

In accordance with yet another aspect provided by this invention, anorganizational structure may be steganographically encoded within aparticular portion of a content-carrying information signal to which theorganizational structure applies—thereby establishing an implicitcorrespondence between the organizational structure and theidentification and/or extent and/or semantics of the information contentto which the organizational structure applies. The correspondence may,for example, include explicit components (e.g., internally statedstart/end points), with the storage or other physical associationdetermined by convenience (i.e., it may make sense to put theorganizational structure close to where it is used, in order to avoidseeking around storage media to find it).

In accordance with yet another aspect provided by this invention,pointers can be steganographically encoded into parts of an informationsignal stream that has little excess available bandwidth. Such pointersmay be used, for example, to direct an electronic appliance to portionsof the information signal stream having more available bandwidth forsteganographic encoding. Such pointers may provide improvedsteganographic decode access time—especially, for example, inapplications in which the information signal stream is stored orotherwise available on a random access basis.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other features and advantages provided by this invention maybe better and more completely understood by referring to the followingdetailed description of presently preferred example embodiments inconjunction with the drawings, of which:

FIG. 1 shows a virtual distribution environment providing steganographicencoding of digital rights management control information;

FIGS. 1A-1E show example electronic appliances embodying aspects of thisinvention;

FIG. 2 shows an example of how electronic control information can besteganographically encoded within an image;

FIG. 3 shows an example rights management component providing asteganographic decoding function;

FIG. 4 shows an example of how steganographically encoded electroniccontrol signals can be extracted and used for digital rights management;

FIGS. 5A-5D show example techniques for enforcing steganographicallyencoded rights management control information;

FIGS. 5E-5F show example “end to end” protected distribution systemsprovided in accordance with the invention;

FIG. 6 shows an example of multiple sets of digital rights managementcontrol information steganographically encoded onto different parts ofthe same information signal stream;

FIG. 7A shows an example detailed steganographic encoding process;

FIG. 7B shows an example detailed steganographic decoding process;

FIG. 8 shows an example frequency domain view of an examplesteganographic signal encoding technique;

FIG. 9 shows an example use of a variable steganographic encoding rateto avoid exceeding channel bandwidths;

FIGS. 10 and 10A show how steganographically encoded pointers can beused to minimize access times to control signals steganographicallyencoded onto information signal streams available on a random accessbasis;

FIG. 11 shows an example steganographically encoded organizationalstructure;

FIG. 12 shows an example electronic appliance architecture havingelectronic rights management capabilities based at least in part onsteganographically encoded control information;

FIGS. 13 and 13A show example control steps that may be performed by theFIG. 12 appliance;

FIG. 14 shows an example steganographic refresh arrangement; and

FIGS. 15A-15F show example distribution systems using steganographicencoding of rights management control information along at least one legof an information distribution path.

DETAILED DESCRIPTION OF PRESENTLY PREFERRED EXAMPLE EMBODIMENTS

FIG. 1 shows an example Virtual Distribution Environment (VDE) 50employing steganography to deliver electronic digital rights managementcontrol information over an insecure (e.g., analog) communicationschannel.

In this example, a provider 60 delivers an information signal 70 tomultiple electronic appliances 100(1), . . . , 100(N). In thisparticular example, provider 60 is shown as being a televisionbroadcaster that delivers an analog television information signal 70over a wireless or cable communications path, and appliances 100(1), . .. , 100(N) are shown as being home color television sets 106. As madeclear by FIGS. 1A-1E, the present inventions may be used by a variety ofdifferent types of electronic appliances 100 receiving a variety ofdifferent types of information signals via a variety of different typesof communications channels.

In the FIG. 1 example, provider 60 steganographically encodes electronicrights management control information 126 into the information signal70. This control information 126 is represented in this diagram as atraffic light because it may define permitted and/or requiredoperation(s), and consequences of performing or failing to perform suchoperations. For example, control information 126 could specify that aviewer or class of viewers has permission to watch a particular program,is forbidden to watch a program, or may watch a program only undercertain conditions (for example, based on paying a certain amount, beingover a certain age, etc.). In this example the control information 126is shown as being packaged within an electronic “container” 136.Container 136 (which in at least one example is provided bysteganographic encoding techniques) is used to protect the integrity ofthe control information 126.

The provider 60 encodes the electronic rights management controlinformation 126 onto information signal 70 using steganographictechniques that make the control information both:

-   -   substantially invisible, and    -   substantially indelible.

The control information 126 is substantially indelibly encoded because,in this example, it is substantially inextricably intertwined with thetelevision images and/or sound—and can't easily be eliminated frominformation signal 70 without destroying the images, sound or otherinformation carried by the information signal. For example,steganographically encoding rights management control information willgenerally survive compression and decompression of a digitized analogsignal, and will also survive repeated analog/digital/analog conversionsequences.

Even though the steganographically encoded control information 126 issubstantially indelible, the television viewer is not bothered by thesteganographically encoded information because the steganographicallyencoded rights management control information is, in this example, alsoencoded substantially invisibly. In fact, the viewer may not be able tosee the steganographic control information at all—and it may have noeffect whatsoever on his or her viewing experience (other than in termsof the effect is has on associated rights management processes). Thecontrol information 126 is shown in dotted lines on the FIG. 1 screensof television sets 106 to emphasize that the control information issubstantially inextricably intertwined with the television images and/orsounds—and yet can't really be seen or noticed by the television viewer.

FIG. 2 shows an example of how digital control information 126 may beencoded within an image 128 so that, in one particular example, it isboth substantially invisible and substantially indelible. In thisspecific image context, for example, “substantially invisible” may referto the characteristic of the encoded control information as notsubstantially interfering with or adversely affecting the viewer'sexperience in viewing image 128 or otherwise using the content carriedby the information signal 70 and/or that it is difficult to detect usingvarious types of signal processing techniques, for example. For example,invisibility can be a measurable quantity (measured in a number ofprocessor instructions, such as MIPS years, for example), and can berelated to signal processing as opposed to the naked eye. In thiscontext, “substantially indelible” can mean, for example, that theencoded digital control information is substantially inextricablyintertwined with the content information, making it difficult forexample to strip out the encoded digital control information withoutalso damaging or degrading the content. Degree of indelibility may, forexample, be measured by the number of processor instructions required tostrip the information out.

FIG. 2 shows that a slight rearrangement of picture elementconfiguration in a small portion of image 128 is one way tosteganographically encode electronic control information into the imageto provide a substantially indelible, substantially invisible encoding.This encoding may be unnoticeable to the viewer, and yet it may bedifficult to strip out or eliminate without also damaging the image.Steganographically encoding digital control information into theinformation signal 70 may effectively merge, from a practicalstandpoint, the digital control information with the other informationcarried by the signal (for example, television programming or othercontent). The steganographic techniques make it difficult for someone tointentionally or unintentionally eliminate the encoded controlinformation without damaging the content, but may (in one example)nevertheless hide the encoded control information so that it does notunduly detract from the content.

Since indelibility of the steganographic encoding provides persistence,indelibility may be more important than invisibility in at least someapplications. For example, it may be desirable in some applications touse a shared secret to decode and then remove thesteganographically-encoded control information 126 before presenting theinformation signal (or its content) to the user. The steganographicallyencoded information need not be particularly invisible in this scenario.Even though someone with knowledge of the shared secret can remove thesteganographically encoded information, it may nevertheless remainsubstantially indelible to anyone who doesn't know the shared secretrequired to remove it.

Organization Structures

FIG. 1 shows that control information 126 may be packaged within one ormore organizational structures such as secure digital containers 136.Containers 136 may be, for example, of the type described in the Ginteret al. patent specification in connection with FIGS. 17-26B. Theorganizational structure(s) may identify, implicitly or explicitly, thecontent the electronic controls apply to. The organizationalstructure(s) may also define the extent of the content, and semantics ofthe content.

The organizational structure(s) may be encoded in such a way that theyare protected for purposes of secrecy, authenticity and/or integrity.The employed steganographic technique may provide such protection, oranother security technique may be used in conjunction with steganographyto provide a desired or requisite degree of protection depending on theapplication. Containers 136 may, for example, use mathematicaltechniques called “encryption” that help guarantee the integrity and/orsecrecy of the control information 126 they contain.

Example Rights Management Component

Each of the FIG. 1 example appliances 100 may include a electronicdigital rights management component 124. Rights management component 124may, for example, comprise one or more tamper-resistant integratedcircuit “chips”. Components 124 may, for example, be of the general typedescribed in detail at FIG. 9 and following of the Ginter et al. patentspecification. Briefly, Ginter et al. describes a Virtual DistributionEnvironment (“VDE”) including multiple electronic appliances coupledtogether through a communications capability. Each electronic appliancehas such a secure, tamper-resistant “protected processing environment”in which rights management processes may securely take place. TheVirtual Distribution Environment delivers digital control information tothe protected processing environments by packaging the controlinformation within secure electronic digital containers. This deliveredcontrol information provides at least part of the basis for performingelectronic rights management functions within the protected processingenvironments.

The ability to securely deliver digital control information to suchprotected processing environments as embodied with components 124 isimportant at least because it increases flexibility and enhancesfunctionality. For example, different digital control information can bedelivered for the same or different electronic content. As one specificexample, one set of rules may apply to a particular television program,another set of rules might apply to a particular film, and a stilldifferent set of rules could apply to a particular musical work. As yetanother example, different classes of users of the same electroniccontent can receive different control information depending upon theirrespective needs.

Rights management components 124 are able to steganographically decodethe control information 126 carried by the information signal 70.Components 124 use the decoded control information 126 to electronicallymanage rights. For example, components 126 may use the decoded controlinformation 126 to control how the images and/or sound carried byinformation signal 70 may be used.

In one example, digital rights management component 124 may comprise orinclude one or more integrated circuit chips as shown in FIG. 3. TheFIG. 3 example rights management component 124 includes ananalog-to-digital converter 130, a steganographic decoder 132, and arights management processor 134. Rights management processor 134 mayinclude or comprise a protected processing environment 138 as describedin Ginter et al. FIGS. 8-12, for example, providing a tamper-resistantexecution environment for effecting the operations provided byelectronic controls 126. Rights management component 124 may alsoinclude a steganographic encoder and a digital-to-analog converter (notshown).

The analog-to-digital converter (ADC) 130 shown in FIG. 3 takes theincoming information signal 70 and—if it is in analog form—converts itto a digital signal (see FIG. 4, step “A”). Steganographic decoder 132obtains the digital control information 126 from the resulting digitalsignal (FIG. 4, step “B”). As mentioned above, digital controlinformation 126 may define permitted and/or required operation(s) on thecontent carried by signal 70, and may further define consequences ofperforming and/or failing to perform such operations. Rights managementprocessor 134 may manage these rights and/or permissions and associatedconsequences (FIG. 4, step “C”).

Example Electronic Appliances

The present inventions may be used with all sorts of different kinds ofelectronic appliances 100 each of which may include a rights managementcomponent 124. FIGS. 1A-1E show various example electronic appliances100 embodying aspects of the present invention. For example:

-   -   FIG. 1A shows an example mediaplayer 102 capable of playing        Digital Versatile Disks (DVDs) 104 on a home color television        set 106. For example, media player 102 may provide analog output        signals to television set 106, and may also process digitized        video and/or audio analog signals stored on optical disk 104.        Rights management component 124A provides digital rights        protection based on steganographically encoded controls 126.    -   FIG. 1B shows an example set top box 108 that can receive cable        television signals (for example, via a satellite dish antenna        110 from a satellite 112) for performance on home television set        106. Set top box 108 shown in FIG. 1B may receive television        signals from antenna 110 in analog scrambled or unscrambled        form, and provide analog signals to television 106. Rights        management component 124B provides digital rights protection        based on steganographically encoded controls 126.    -   FIG. 1C shows an example radio receiver 114 that receives radio        signals and plays the radio sound or music on a loud speaker        116. The radio receiver 114 of FIG. 1C may receive analog radio        signals, and provide analog audio signals to loud speaker 116.        Rights management component 124C provides digital rights        protection based on steganographically encoded controls 126.    -   FIG. 1D shows an example video cassette recorder 118 that can        play back video and sound signals recorded on a video cassette        tape 120 onto television 106. In FIG. 1D, the video tape 120 may        store video and audio signals in analog form, which VCR 118 may        read and provide to television 106 in analog form. Rights        management component 124D provides digital rights protection        based on steganographically encoded controls 126.    -   FIG. 1E shows an example television camera that can capture        video images and produce video signals for recording on a video        cassette tape 120 and play back on television set 106. The FIG.        1E camcorder 122 may generate analog video and audio signals for        storage onto video tape 120, and/or may provide analog signals        for processing by television 106. Rights management component        124E provides digital rights protection based on        steganographically encoded controls 126.

Example Rights Management Enforcement Techniques

Different rights holders want different types of rights management andcontrol. For example, some rights holders may be completely satisfiedwith a relatively simple “copy/no copy/one copy” rights managementcontrol model, whereas other rights holders may desire a richer, morecomplex rights management scheme. The present inventions flexiblyaccommodate a wide variety of electronic rights managementtechniques—giving rightsholders extreme flexibility and programmabilityin defining, for example, commerce and rights management models that farexceed the simple “copy/no copy, one copy.” Assuming a closed appliance,that is, one lacking at least an occasional connection to a paymentmethod (e.g., Visa, MasterCard, American Express, electronic cash,Automated Clearinghouses (ACHs) and/or a Financial Clearinghouse thatserves as the interface for at least one payment method), the followingare non-limiting examples of steganographically encoded rights controlsand associated consequences that can be accommodated by the presentinvention:

-   -   Limiting use of a given property to a specified number of times        this property can be used on a given appliance;    -   Prohibiting digital to analog and analog to digital conversions;    -   Ensuring that one analog or digital appliance will communicate        the protected property only to another appliance that is also        VDE enabled and capable of. enforcing the controls associated        with that property;    -   Time-based rental models in which a consumer may “perform” or        “play” the property an unlimited number of times in a given        interval (assuming the appliance has a built-in secure time        clock, can operatively connect itself to such a clock, or        otherwise receive time from a reliable source);    -   Enforcing an expiration date after which the property cannot be        performed (also assuming access to a reliable time source);    -   Associating different control sets with each of several        properties on a single physical media. In one example, a        “trailer” might have unlimited copying and use associated while        a digital film property may have an associated control set that        prevents any copying;    -   Associating multiple control sets with a given property        regardless of media and whether the appliance is closed or has        an occasionally connected communications “backchannel.”

An even more flexible and diverse array of rights controls andassociated consequences are enabled by the present inventions if atleast one appliance is connected to some form of communications“backchannel” between the appliance and some form of payment method.This backchannel may be a telephone call, the use of a modem, a computerdata network, such as the Internet, a communications channel from asettop box to the head end or some other point on a cable TVdistribution system, or a hybrid arrangement involving high bandwidthdistribution of analog properties with a slower return channel, a phoneline and modem—just to name a few examples. Non-limiting examples ofsuch more rights controls and associated consequences enabled by thepresent invention include the following:

-   -   Associating with a given property in analog format new,        independently delivered controls obtained from a rightsholder or        other authorized source;    -   A broad range of usage-based pricing models, including        pay-per-view or pay-per-use;    -   Creating permissions enabling excerpting of properties in analog        formats, maintaining persistent control over those excerpts, and        charging for those excerpts;    -   Pay-per-use models in which a customer pays a specified price        for each use of the property and/or different unit prices        depending on the number of uses. In one example, the customer        might pay $3.99 for the first viewing and $2.99 for each        subsequent viewing; and,    -   Controls that prevent an analog property being converted to        digital format and then being transmitted or communicated except        in a container with controls and/or with a pointer to a source        of controls, that apply in a digital environment.

FIGS. 5A-5D show some examples of how rights management component 124can enforce steganographically encoded digital rights managementcontrols.

In the FIG. 5A example, rights management component 124 controls anon/off switch 140 based on steganographically encoded electroniccontrols 126. Component 124 turns switch 140 on (for example, to allowthe analog television signal to pass to television set 106) whenelectronic controls 126 permit, and otherwise opens (turns off) switch140 to prevent the analog signal from reaching the output.

In a more secure example, the incoming analog signal is scrambled, andthe FIG. 5A on/off switch 140 is replaced by a FIG. 5B descrambler 142of conventional design. The descrambler 142 descrambles the analog inputsignal to provide a descrambled output under control of rightsmanagement component 124. Rights management component 124 allowsdescrambler 142 to descramble the analog signal only under conditionsspecified by electronic controls 126 that the component 124 obtains fromthe analog input signal. Scrambling the analog signal gives the rightsmanagement component 124 a relatively secure way of enforcing electroniccontrols 126—since the rights management component can prevent thedescrambler from operating unless conditions set by the controls aresatisfied. The rights management function and the descrambling functionmay be integrated into a single component in which the descramble anddecrypt functions of the rights management component are essentiallyserving the same function, but may still be distinct to account forspecialized approaches to descrambling that may not be sufficientlystrong or interoperable with other environments to use generally. Ifthey are separate components, the data path between them should beprotected (for example, by ensuring that both components are in a tamperresistant enclosure, or using secure authentication and key exchange tosend the descrambling sequence to the descrambler).

FIG. 5C shows how digital certificates may be used to enforcesteganographically encoded electronic controls 126. In this example,appliance 100A outputs content to another appliance 110D only ifappliance 100D has a rights management component 124D that can enforcethe electronic controls 126. In this example, there may be a “handshake”between the content supplying appliance 100A and the content receivingappliance 100D sufficient to ensure the content supplying appliance thatthe content receiving appliance will enforce the electronic controls126. For example, the supplying appliance 100A's rights managementcomponent 124A may require the receiving appliance 100D's rightsmanagement component 124D to present a digital certificate 199 attestingto the fact that the receiving appliance 100D has a rights managementcomponent 124 fully capable of securely enforcing electronic controls126. Receiving appliance 110D could present this digital certificate 199by steganographically encoding it within an analog signal it provides tothe supplying appliance over an analog signal channel for example (theanalog signal channel could be the same one the supplying appliance willuse to deliver the steganographically encoded content). If a digitalchannel is available, the handshake can be over a digital link betweenthe two appliances using, for example, secure authentication techniquesdisclosed in Ginter et al. and/or for example in Schneier, AppliedCryptography (2d Ed. Wiley 1996) at page 52 et seq.

FIG. 5D shows that rights management component 124A can enforceelectronic controls 126 by marking the content through “fingerprinting”and/or “watermarking” prior to releasing the content to a device thatdoesn't have a rights management component 124. See Ginter et al. patentspecification, FIGS. 58A-58C. Such fingerprinting could involve usingsteganographic techniques to fingerprint the content. For example, amovie delivered using “conventional” containers as disclosed in Ginteret al. could use steganographically encoded containers “on the way” tothe display device. Furthermore, it could include the identity of theuser, etc. as well as the control information appropriate for thedevice. Another case could be text sent to a printer, using differentsteganographic encoding techniques such as line and/or charactershifting.

End to End Protection

FIGS. 5E-5F illustrate how the persistent association with contentprovided by steganographically encoded electronic rights managementcontrol information 126 provides “end to end” protection within anarbitrary information signal distribution system—irrespective of theprocesses the information signal is subjected to as it travels to itsfinal destination.

FIG. 5E shows an example of how the present inventions can be used tomaintain end-to-end rights management protection over content initiallydistributed in an analog signal format. FIG. 5F shows an example of howthe present invention can be used to maintain end-to-end rightsmanagement protection over content initially distributed in digitalform.

In the FIG. 5E example, an analog signal transmission site (e.g., aradio or television broadcaster) transmits an analog signal Asteganographically encoded with an organizational structure 136including electronic controls 126. This analog signal A may be receivedby an electronic appliance 100A having a rights management component124A as described above. Appliance 100A may, for example, convert thesignal into digital and/or digitized format, and store the digitizedversion of the signal onto a digital storage medium 104. Electronicappliance 100A may play back the recorded digitized signal, convert thesignal back to analog form, and deliver the analog signal A to a furtherelectronic appliance 106B. In this example, electronic appliance 106Balso has a rights management component 124B.

The steganographic techniques provided by the present invention ensurethat the electronic controls 126 persist in the signal A delivered fromappliance 100A to appliance 106B—and from appliance 106B to still otherappliances. Because of the substantial indelibility characteristics ofthe steganographically encoded control information 126, this informationpersists in the signal as stored on recording medium 104, in copies ofthe recorded signal produced by replaying the medium, and in furtherdownstream versions of the signal.

This persistence will, for example, survive conversion from analog todigital format (e.g., sampling or “digitizing”), storage, and subsequentconversion from digital to analog format. For example, because thesteganographically encoded control information 126 is substantiallyindelibly, substantially inextricably intertwined and integrated withthe information signal A, the digitized version of the informationsignal that appliance 100A records on medium 104 will also contain thesteganographically encoded control information 126. Similarly, whenappliance 100A plays back the recording from medium 104, it willreproduce information signal A along with the steganographically encodedcontrol information 126. The steganographically encoded controlinformation 126 thus persists irrespective of digitization (or otherprocessing) of signal A. In some cases, lossy compression techniquesused on the data may remove high frequency noise—thereby potentiallydamaging the steganographic channel. When these lossy compressiontechniques are used or may be encountered, the steganographic encodingfunction should be matched to the compression algorithm(s) usingconventional signal analysis techniques to avoid this consequence.

Similarly, appliance 106B may output further copies or versions ofsignal A in analog form and/or digital form. Because of its inherentlypersistent characteristics, the steganographically encoded controlinformation 126 will be present in all subsequent versions of the signaloutputted by appliance 106B—be they in analog format, digital format, orany other useful format.

Degrading a digital signal carrying control information is fatal—therights management system typically may no longer function properly ifeven a single bit is altered. To avoid this, the preferred embodimentprovides redundancy (repeating pointers and the organizationalstructures and/or any control information incorporated into theorganizational structures), and also uses conventional error correctioncoding such as, for example, Reed-Solomon (or similar) error correctingcodes. Additionally, because the steganographically encoded controlinformation 126 is substantially inextricably intertwined with thedesired content carried by information signal A, any process thatdegrades the steganographically encoded control information 126 willalso tend to degrade the information signal's desired content. Althoughthe steganographically encoded information may degrade (along with thecontent) in multi-generation “copies” of the signal, degraded copies maynot be commercially significant since the information content of thesignal will be similarly degraded due to the substantially inextricableintertwining between the steganographically encoded control information126 and the content carried by signal A. The refresh circuit shown inFIG. 14 with appropriate error correcting capabilities is one way toprevent the steganographically encoded information from being degradedeven if the rest of the information the signal carries becomes degraded.

The FIG. 5F example shows content being initially distributed in digitalform over a network to an electronic appliance 100J such as a personalcomputer. Personal computer 100J may convert the digitally deliveredcontent to an analog signal A for distribution to other appliances 106B,100A. Personal computer appliance 100J may include a rights managementcomponent 124J that ensures, based on controls 126, that appliance 100Jdoes not release a version of the content associated with controls 126that is not protected by the controls. In this example, rightsmanagement component 124J is capable of steganographically encoding theanalog signal A with the control information 126 (e.g., it may performthe processes shown in FIG. 7A below). Rights management component 124Jenforces controls 126, at least in part, by ensuring that any analogversion of the content associated with controls 126 issteganographically encoded with those controls. Further “downstream”.appliances 106B, 100A may each include their own rights managementcomponent 124 for use in interacting with steganographically encodedcontrols 126.

Example Control Information

FIG. 6 shows that a particular information signal 70 may be encoded withmany different containers 136 and associated rights management controlsets 126. For example, different portions of an information signal 70may be associated with different control information 126. In thisexample of a movie 270:

-   -   a first “trailer” 272 may be associated with control information        126(1),    -   a second trailer 274 may be associated with control information        126(2),    -   a title section 276 may be associated with control information        126(3),    -   the first five minutes of the movie may be associated with        control information 126(4), and    -   the rest of the movie may be associated with control information        126(5).

Control information portions 126(1), 126(2), 126(3), 126(4) and 126(5)may all be different. For example, control information 126(1) may permitthe user to copy trailer 272, whereas control information 126(4) mayprohibit the user from copying the first five minutes 278 of the film.

As shown in FIG. 6, multiple, identical copies of control information126(5) may be steganographically encoded onto the information signal 70.For example, control information 126(5) could be encoded once per minuteonto the rest of movie 280. This redundancy allows a media player 102 orother electronic appliance 100 to rapidly obtain a copy of the controlinformation 126(5) no matter where the user begins watching or playingthe movie 270, and also helps ensure that transmission errors will notprevent the rights management component 124 from recovering at least one“good” copy of the organizational structure.

Example Steganographic Encoding and Decoding Processes

FIGS. 7A and 7B show example overall steganographic encoding anddecoding processes, respectively. The FIG. 7A process may be used tosteganographically encode digital control information onto an analogsignal, and FIG. 7B performs the inverse operation of steganographicallydecoding the control information from the analog signal. Generally, theFIG. 7A process may be performed at a supply point, and the FIG. 7Bprocess may be performed at a usage point. An electronic appliance 100can be both a supply point and a usage point, and so it may perform boththe FIG. 7A process and the FIG. 7B process.

Referring to FIG. 7A, the analog information signal 70 inputted to thesteganographic encoding process may be any sort of information signalsuch as, for example, the analog signal shown in Graph A1. Aconventional analog-to-digital conversion block 402 may be used, ifnecessary, to convert this analog input signal to a digitized signal(see Graph A2). A spectral transform block 404 may then be used totransform the digitized information from the time domain to thefrequency domain. Spectral transform block 404 may be any conventionaltransformation such as, for example, a Fast Fourier Transform (FFT) or aWalsh Transform. An example of the resulting spectral information isshown in the A3 graph.

A steganographic encode block 406 may be used to steganographicallyencode digital control information 126, in clear text form and/or afterencryption by a conventional digital encryption block 414 based on anencryption key Key, Steganographic information can be combined with apseudo-random data stream (e.g. exclusive-or'd into the output of a DESengine)—in effect shuffling around the noise in the signal rather thanreplacing noise with the signal, per se. When protection is desired, thevalues in the pseudo-random stream can be protected by encryption (e.g.the key that initializes the DES engine should be protected). When thesteganographic channel is “public” (e.g., unencrypted), the streamshould be readily reproducible (e.g. by using one of a preset collectionof values shared by every device). A small portion (a “publicheader”—see Ginter et al.) is always detectable using a shared presetvalue (that does not need to be protected, distinguishing it from theprivate header keys), may be provided to ensure that the rightsmanagement technology can be activated properly. Since the rightsmanagement component 124 at the receiving side needs to know how todescramble the signal, there normally will be an indication in the“public header” that names a key that will be used to unlock the privateheader (and so on, as described, for example, in Ginter et al.). Somepublicly available, agreed upon preset values may be used to extract the“public header” information from the steganographically encoded channel.

Steganographic encode block 406 may be any conventional steganographicencoding arrangement capable of steganographically encoding a digitalsignal onto information signal 70. Steganographic encode step 406 may bebased on a key K_(c)—allowing the same basic steganographic encoding anddecoding transformations to be used by a wide variety of differentappliances while still maintaining individuality and secrecy through theuse of different steganographic keys.

In one example, the steganographic encoding step 406 may introduce the(encrypted) digital control information into the high frequency spectrumportion of the spectrally transformed information signal 70. Thespectrally transformed signal with steganographic encoding is shown inthe FIG. 7A Graph A4, and is shown in more detail in FIG. 8. As FIG. 8shows, the steganographic encoding may affect the higher order frequencycomponents of the spectrally transformed signal (see dottedperturbations in the fourth, fifth, sixth, seventh and eighth ordercomponents in FIG. 8). The steganographic encoding may add to and/orsubtract from the amplitudes of these higher order components. Theeffect of introducing high frequency steganographically encoded signalcomponents may be to mask the steganographic encoding within the randomhigh frequency noise inherently provided within information signal70—thereby providing substantial invisibility and substantialindelibility.

The amount of amplitude modification performed by steganographic encodestep 406 may be limited in this example to ensure that the resultingsteganographically encoded signal does not exceed the available channelbandwidth. See, for example,

-   -   J. Millen, “Covert Channel Capacity,” IEEE Symposium on Security        and Privacy (1987).    -   R. Browne, “An Entropy Conservation Law for Testing the        Completeness of Covert Channel Analysis,” Fairfax 94, pp 270-281        (1994).    -   Moskovitz et al., “The Channel Capacity of a Certain Noisy        Timing Channel,”, IEEE Trans. on Information Theory v IT-38 no.        4, pp. 1330-43, (1992).    -   Venkatraman, et al., “Capacity Estimation and Auditability of        Network Covert Channels,”, Oakland 95, pp. 186-298.

The following equations show the relationship between total bandwidth,bandwidth available for steganographic encoding, and the data rate ofthe steganographically encoded signal:

$\begin{matrix}{S = {\int_{ta}^{tb}{{B(t)}\ {\mathbb{d}t}}}} & (1) \\{\cong {\sum\limits_{i = a}^{b}\;{{B(i)}\Delta\; t}}} & \left( {1A} \right)\end{matrix}$where Δt=t_(n+1)−t_(n), and

-   B is a function of time in bits/second.

In the above expressions, the function S corresponds to an area under acurve resulting from the product of B (bandwidth) and t (time). Theparameter delta t refers to the “granularity” of the analog-to-digitalconversion (i.e., 1/sampling rate).

FIG. 9 shows an example plot of information signal bandwidth versustime. The total bandwidth available is limited by the bandwidth of thetransmission channel—including the bandwidth of the storage medium (ifany) used to deliver the signal, and the bandwidth of the reproductionequipment. Since the total bandwidth depends on the inherentcharacteristics of the transmission channel used to communicateinformation signal 70, it is typically a fixed constant. FIG. 9 showsthat the bandwidth actually used by the information signal 70 typicallyvaries with time. For example, although somewhat counterintuitive, themore complex an image, the more noise is typically available for“shuffling around” to create a steganographic channel. Of course, thisisn't always true—a highly intricate geometric pattern may have verylittle noise available for encoding, and a simple picture of a cloud mayhave a great deal of noise available.

Steganographic encode block 406 can use an encoding rate andcharacteristic that ensures the steganographically encoded signalbandwidth doesn't exceed the total bandwidth available in thecommunication channel. Typically, the amount of bandwidth available forsteganographic encoding may be on the order of on the average of 0.1% ofthe total transmission channel bandwidth—but as mentioned above, thisbandwidth available for steganographic encoding may be unequallydistributed with respect to time within the information signal stream 70and may depend on the content of the information signal.

In this example, steganographic encode block 406 analyzes the content(e.g., by performing statistical weighted averaging), and provides aresponsive variable steganographic encoding rate. For example,steganographic encoding block 406 can use a high data rate duringexample time periods “II” and “IV” in which the information signal 70has characteristics that allow high steganographic rate encoding withoutthe resulting signal exceeding the available overall channel bandwidth.Encoding block 406 can use a low data rate during time periods “I” and“III” in which the information signal 70 has characteristics that do notallow high data rate steganographic encoding without exceeding availableoverall channel bandwidth. Steganographic encoding block 406 may use anynumber of different variable rates to accommodate differentrelationships between information signal 70 characteristics andavailable channel bandwidth.

Referring again to FIG. 7A, the steganographically encoded spectralinformation outputted by steganographic encode block 406 may besubjected to an inverse spectral transform 408. Inverse spectraltransform 408 in this example may perform the inverse of the transformperformed by step 404—outputting a version of the digitized time domainsignal shown in Graph A2 but now bearing the steganographically encodedinformation (Graph A5). The digital control informationsteganographically encoded by block 406 may be substantially indelibleand substantially invisible with respect to the Graph A5 signal—that is,it may be very difficult to eliminate the steganographically encodedinformation and it may also be very difficult to discern it.

This signal may be further scrambled and/or encrypted (e.g., based on ascrambling and/or encryption key Key_(d)) before being converted toanalog form (shown in Graph A6) by a conventional digital-to-analogconversion block 412 (if necessary). Signal scrambling may beindependent of steganographically encoded control information. Forexample, a good way to support existing devices is to not scramble thesignal, and to use legislative means to ensure that each new devicemanufactured is equipped with rights management technology.Scrambling/encrypting of content, can be used to enforce use of rightsmanagement. If legislative means can enforce the use of rightsmanagement technology, encryption or scrambling of content may not benecessary (although a decision to provide cryptographic protection forthe control information is independent of this factor and must beevaluated in light of protecting the rights management system). Rightsholders can choose an enticement technique(s) based on their businessmodel(s). The benefit of scrambling is that it provides technical meansfor enforcing rights management. The benefit of unscrambled content issupport of hundreds of millions of devices in the installed base—withthe promise that new devices (potentially including computers) willenforce the control information even though they don't “have to” from atechnical perspective.

The resulting steganographically encoded information signal 70 may thenbe transmitted over an insecure communications channel.Digital-to-analog conversion step 412 may be omitted if a digitalcommunications channel (e.g., an optical disk, a digital satellite link,etc.) is available to deliver the signal.

FIG. 7B shows an example inverse process for recovering digital controlinformation 126 from the steganographically encoded information signal70. In this recovery example, the steganographically encoded analogsignal is converted to a digitized signal (if necessary) by ananalog-to-digital conversion step 402′ and decrypted/descrambled (ifnecessary) by a decryption/descrambling block 422′ to yield a facsimileof the inverse spectral transform block 408 output shown in FIG. 7A. Inthis FIG. 7B example, the analog-to-digital conversion block 402′ is theinverse operation of FIG. 7A, block 412, and the decrypt/descrambleblock 422′ is the inverse of the FIG. 7A scramble/encrypt block 410.

The resulting digitized signal provided by FIG. 7B block 422′ isspectrally transformed by step 404′ (this may be the same spectraltransform used in FIG. 7A, block 404) to yield a steganographicallyencoded spectral signal A3. Steganographic decode block 424 may performthe inverse operation of the FIG. 7A steganographic encode block 406based on the same steganographic key Key, (if a key-based steganographicencoding/decoding transformation is used). The output of steganographicdecode block 424 may be decrypted by block 426 (the inverse of FIG. 7Aencrypt block 414 based on key Key_(s)) to provide recovered digitalcontrol information 126. The resulting control information 126 may beused for performing electronic rights management functions Required keysmay be delivered in containers and/or using the key distributiontechniques and device initialization approaches disclosed in Ginter etal., for example.

Example Control Information Arrangements

In a further example shown in FIGS. 10 and 10A, steganographic encodeblock 406 may encode control information organizational structures suchas secure containers (see Ginter et al., FIGS. 17-26B and associatedtext) during times when the content bandwidth is low relative to thetotal available bandwidth (see FIG. 10 regions II and IV), and may notattempt to encode such organizational structures during times when thecontent bandwidth is high relative to the total available bandwidth (seeFIG. 10, regions I, III). In this way, steganographic encode block 406may maximize the total bandwidth use without causing thesteganographically encoded signal to exceed available bandwidth. As anoptimization for certain applications, steganographic encode block 406may encode “pointers” or other directional information into theinformation signal 70 during times when the content is such that itdoesn't allow high data rate steganographic encoding of organizationalstructures 136. Multiple pointers and multiple “pointed to” locationscan also help provide redundancy.

This particular FIG. 10 example involving steganographic encoding ofpointers 800 may be especially suited for content delivery orpresentation on random access storage media such as optical disks. Usingsuch random access media, a content handling device may be able torapidly “seek” to the place where an organizational structure is storedat a higher recorded bandwidth and then read the organizationalstructure at this higher bandwidth (See FIG. 10A). For these examplearrangements, steganographic encode block 406 in this example encodes,during periods when the content is such that it is not possible tosteganographically encode organizational structures, pointers 800 thatdirect the content handling device to one or more places where theorganizational structure appears in the content stream. In one example,pointers 800 might encode the location(s) on a storage medium (e.g., anoptical disk 104—see FIG. 10A) at which the closest organizationalstructure is stored.

An optical disk player 102 with random access capability may “seek” tothe place at which the closest organizational structure 136 is stored onthe disk 104, and rapidly read the organizational structure off of thedisk in less time than might be required to read an organizationalstructure that steganographic encode block 406 encodes at a lower datarate during times when the content bandwidth occupies most of theavailable channel bandwidth. In such arrangements, the process ofreading a pointer 800, “seeking” to a position on the medium specifiedby the pointer, and then reading an organization structure 136steganographically encoded at a high data rate may provide overallfaster access times than if the organizational structure was itselfencoded at a lower data rate within the parts of the information signalstream used in this example to encode only pointers.

FIG. 11 shows an example organizational structure 136 suitable forsteganographic encoding similar to that shown in FIG. 17 of theco-pending Ginter et al. application. In the case of container 136 withcontrols for an analog property, the organizational structure mayinclude one or more permissions records 136 d providing control sets 136e providing electronic controls especially for an analog device(s). Thepermissions record 136 d may also provide a reference 136 f at least onelocation or other external source for additional controls. Thisreference may be to an Internet “Uniform Resource Locator” (URL), forexample. The organizational structure 136 may optionally include acontent block 136 g providing digital content subject to the controls.In this example, organizational structure 136 is encased in a protective“wrapper” 136 x provided by the steganographic technique used to encodethe organizational structure 136, digital encryption techniques, and/ora combination of the steganography and encryption. This protectivewrapper 136 x is used to ensure that the organizational structure 136cannot be tampered with and maintains its integrity. Wrapper 136 x mayalso provide a degree of confidentiality if required.

Detailed Example Electronic Appliance Architecture

FIG. 12 shows an example detailed internal architecture for an exampleelectronic appliance 100 such as optical disk player 102. In thisspecific example, rights management component 124 may be atamper-resistant integrated circuit including internal microprocessor200, flash memory 202 and cryptographic engine 204 (see Ginter et al.FIGS. 9-15B and associated text for a more detailed internal view of anexample tamper-resistant rights management component 124 and a“protected processing environment” 138 it provides).

A main system bus 206 may couple rights management component 124 to amain system microprocessor 208 and various system components such as,for example, a CD-ROM decoder 210, a control and audio block 212, avideo decoder 214, a digital output protection block 216, and acommunications system 218. In this example, main microprocessor 208controls the overall operations of appliance 100, with rights managementcomponent 124 performing security-related functions such as-rightsmanagement and steganographic decoding.

In the FIG. 12 example appliance 102, an optical pickup 220 readsinformation from optical disk 104 and provides it to RF amplifier 222.RF amplifier 222 provides its output to digital signal processor (DSP)224, which processes the output in a conventional manner and alsocontrols the orientation of the optical disk 104 relative to opticalpickup 220 via a driver 226. DSP 224 coordinates with a conventionalCD-ROM decoder 210 to provide decoded digitized video and audioinformation. Decoder 210 operates in conjunction with a buffer memory228, and may also cooperate with cryptographic engine 204 to ensure thatany encrypted video information is decrypted appropriately.

The video output of CD-ROM decoder 210 may be decompressed by MPEG-2video decoder 214 and applied via an NTSC and/or PAL encoder 230 totelevision 106. (In another example, the output could be in anon-interlaced format such as RGB rather than in interlaced formats suchas NTSC and PAL.) Meanwhile, control and audio block 212 (which mayoperate in conjunction with its own buffer memory 232) may receivedigitized audio information recorded on optical disk 204 via DSP 224 andCD-ROM decoder 210. Control and audio block 212 may provide this audiooutput to audio processing block 234 for output to loudspeakers 116.Control and audio block 212 may also provide an interface to the uservia an infrared sensor 236 (for a remote control, for example),front-panel user controls 238 and/or an LED display 240.

In this example, security microprocessor 200 within rights managementcomponent 124 receives the digitized video and/or audio that DSP 224reads from optical disk 104 via pickup 220 and RF amp 222. Securitymicroprocessor 200 steganographically decodes this digitized analoginformation signal to recover the digital control information 126encoded onto the information signal. Security microprocessor 200 alsoperforms rights management functions based on the digital controlinformation 126 it recovers. In addition, if desired securitymicroprocessor may remove the steganographic encoding from a receiveddigitized analog signal (since it shares a secret such as thesteganographic encoding key Key with the steganographic encoding point,it can remove the steganographic encoding) and/or steganographicallyencode a signal with received, augmented and/or new rights managementcontrol information.

In this example, microprocessor 200 may selectively control cryptographyengine 204 to decrypt encrypted content provided by optical disk104—thus enforcing the rights management activities provided inaccordance with electronic controls 126. Security component 124 may alsocontrol digital output protection block 216 in accordance with rightsmanagement control information 126—thus, selectively permitting digitalappliance 100 to output content in digital form. Rights managementcomponent 124 may take other steps (e.g., watermarking and/orfingerprinting information before releasing it) to provide a degree ofcopy protection and/or quality degradation to prevent or discouragesomeone from creating an unlimited number of high quality copies of thecontent of optical disk 104. Rules contained in the control informationcan also govern how other parts of the system behave. For example, thecontrol information could specify that no sound can be played unless thecontent is paid for. Another property may specify that certain copyprotection schemes should be turned on in the NTSC encoder. Stillanother might disable the digital outputs of the device altogether, orunless an additional fee is paid.

Rights management component 124 (protected processing environment 138)may, in this particular example, communicate over a network 144 (suchas, for example, the Internet or other data communications path) withother rights management related entities, such as, for example,clearinghouses and repositories. This “back channel” allows rightsmanagement component 124 to, for example, report usage and paymentinformation and/or to retrieve additional rights management controlinformation 126 to augment or supplement the control information itsteganographically decodes.

Example Control Steps

FIG. 13 shows example control steps that may be performed by protectedprocessing environment 138 (e.g., security microprocessor 200) toprovide electronic digital rights protection. The FIG. 13 read/playroutine 300 begins with protected processing environment 138 applyingrules 126—in effect, setting the initial state in which rightsmanagement can occur (FIG. 13, block 302). Protected processingenvironment 138 then reads the output of CD-ROM decoder 310 (FIG. 13,block 304) and obtains steganographically encoded data from the outputstream (FIG. 13, block 306). If protected processing environment 138encounters the beginning of the control information organizationalstructure (“yes” exit to decision block 308), the protected processingenvironment performs an initialization step (FIG. 13, block 310) tobegin receiving new control information 126 and then returns to block302 to again apply current control information (FIG. 13, block 302). If,on the other hand, protected processing environment 138 encounters acontinuation of an organizational structure (“yes” exit to decisionblock 312, FIG. 13), the protected processing environment stores theorganizational structure information it has received (FIG. 13, block314) and turns again to the apply rules step (FIG. 13., block 302).

If protected processing environment 138 encounters a pointer (“yes” exitto decision block 318), then the protected processing environmentdetermines whether it already has received the correspondingorganizational structure pointed to by the received pointer (FIG. 13,decision block 320). The protected processing environment 138 retrievesthe organizational structure if it does not already have it (FIG. 13,block 322)—for example, by controlling DSP 224 to seek to thecorresponding location on optical disk 104 indicated by the pointer, andby reading the organizational structure from the disk beginning at thatdisk location (FIG. 13, block 322).

If protected processing environment 138 has received no organizationalstructures or pointers (“no” exits to each of decision blocks 308, 312,318), then the protected processing environment may determine whetherthere is any bandwidth available to carry control information. Forexample, some types of content stored on optical disk 104 may take upsubstantially all available channel bandwidths so that no bandwidthremains for steganographic encoding. If there is no available bandwidthfor steganographic encoding (“no” exit to decision block 324), then theprotected processing environment 138 may return to the “apply rules”block 302 and repeat steps 304-324 to wait until bandwidth is availablefor steganographic encoding. On the other hand, if there is bandwidthavailable and still no steganographically encoded information hasappeared (“yes” exit to decision block 324, FIG. 13), protectedprocessing environment 138 performs an error handling routine thatprocesses the exception (FIG. 13, block 326) and determines whether theexception is critical (decision block 328). In some cases, protectedprocessing environment 138 will continue to allow the appliance 100 toprocess the content, finding the error to be non-critical (“no” exit todecision block 328). An example of this would be a timer that permitsplaying for a period of time. In other cases (e.g., if the errorconditions indicate that optical disk 104 has been tampered with),protected processing environment 138 may halt processing and return anerror condition (“yes” exit to decision block 328, bubble 329).

FIG. 13A shows example steps that may be performed by the FIG. 13 “applyrules” routine 302. In this example, protected processing environment138 may determine if it has received a complete organizational structureon which to base rights management for the rights being read fromoptical disk 104 (FIG. 13A, decision block 330). If the protectedprocessing environment 138 has not received a complete organizationalstructure (“no” exit to decision block 330), it may disable contentprocessing until it receives a complete organizational structure (FIG.13A, block 332). If protected processing environment 138 has a completeorganizational structure (“yes” exit to decision block 330), itdetermines whether it has the current organizational structure (decisionblock 334). If the current organizational structure is present (“yes”exit to decision block 334), the protected processing environment 138then processes the current operation with respect to the controlinformation embodied in the organizational structure (FIG. 13A, block336). If the protected processing environment 138 does not have thecurrent organizational structure (“no” exit to decision block 334), itdetermines whether it has an organizational structure that has the sameidentification as the current organizational structure (FIG. 13A,decision block 338). The protected processing environment 138 may usethat matching organizational structure as a default (“yes” exit todecision block 338, block 340). Otherwise, protected processingenvironment 138 disables content operations until it receives a currentorganizational structure (“no” exit to decision block 338, block 342).

As mentioned above, protected processing environment 138 may alsoperform any or all of the FIG. 7A steganographic encoding steps, and mayalso or alternatively remove the steganographic encoding from a signalby using a shared secret to generate a steganographic encoding streamand then subtracting that stream from the signal. Such techniques may beuseful, for example, to allow protected processing environment 138 toencode new control information or to change the encoded controlinformation. For example, the steganographically encoded controlinformation might provide a chain of handling and control thatauthorizes certain protected processing environments to change someelements and add new elements to the control information 126. Protectedprocessing environment 138 could:

-   -   steganographically decode the signal using shared secrets to        obtain the control information;    -   modify the control information to the extent authorized by the        control information;    -   remove the steganographic encoding from the signal based on the        shared secret; and    -   steganographically encode the signal with the modified control        information.

Example Refresh Capability

FIG. 14 shows another example electronic appliance arrangement includinga “refresh” capability involving both steganographic decoding andsteganographic encoding. In this example, electronic appliance 100includes a steganographic decoding block 424 as described above plus anadditional steganographic encoding block 406. The appliance 100 mayobtain the digital control information from the content signal, and thenmay “refresh” the extracted information (e.g., using coding techniques,such as, for example, Reed-Solomon decoding based on Reed-Solomon codesapplied to the signal by the steganographic encoding process) to correcterrors and otherwise accurately recover the digital control information.The error-corrected digital control information outputted by refreshdecoder 900 may be applied to a steganographic encoding circuit 406which steganographically encodes the content signal with the refreshedcontrol information.

The FIG. 14 refresh operation could, for example, be performed on aselective basis based on the encoded digital control information itselfFor example, the control information might authorize appliance 100 toredistribute the content signal only under certain conditions—one ofwhich is to ensure that a refreshed steganographic encoding of the same(or modified) digital control information is provided within theredistributed content signal.

EXAMPLES

FIG. 15A shows an example analog signal distribution arrangement 500provided in accordance with this invention. Within arrangement 500, asteganographic encode block 400 encodes an analog information signal Awith rights management control information 126 and associatedorganizational structure(s) 136. The steganographically encodedinformation signal A′ is distributed by various mechanisms to userelectronic appliances 100. For example, the encoded signal A′ may bebroadcast wirelessly over the air by a broadcaster 60A, distributed overa cable television network by a cable television head end 502, and/ordistributed via a satellite communications network 504. Encoded signalA′ may, during the process of being distributed, be converted fromanalog to digital form and back again. For example, the satellite uplink504A may digitize signal A′ before transmitting it to the satellite 504b, and the satellite downlink 504 c may convert the signal back toanalog before providing it to user appliances 100. As explained above,the steganographically encoded control information 126 persists withinthe signal A′ despite conversions between analog and digital formats.

In this example, an example set top box user appliance 108 may receivethe distributed steganographically encoded analog signal A′. Set top box108 may include a rights management component 124 as described above,and may perform rights management operations and/or processes inresponse to and based on steganographically encoded control information126.

Set top box 108 in this example may output the steganographicallyencoded analog signal (or a facsimile of it) to additional userelectronic appliances such as, for example, a television set 106, adigital optical recording device (e.g., DVD-R) 102, and/or a video taperecorder 118. Each of these additional appliances 106, 102, 118 mayinclude a rights management component 124 that performs electronicrights management based on the steganographically encoded controlinformation 126. Any recordings made by recording devices 102, 118 mayalso be steganographically encoded.

FIG. 15B shows another example analog signal distribution arrangement510. In this example, a radio broadcaster 60B broadcasts an analog radiosignal A′ that is steganographically encoded with associated rightsmanagement control information 126 and associated organizationalstructure(s) 136. A wire network 512 such as a cable television systemmay similarly distribute the same or different steganographicallyencoded analog radio signal A′. Broadcaster 60B and/or network 512 maydeliver the steganographically encoded radio signal A′ to a userreceiving appliance 100C such as a FM radio receiver 114. In thisexample, radio receiver 114 has a rights management component 124 thatprocesses and automatically manages rights based on steganographicallyencoded controls 126. In this example, radio receiver 114 may (ifpermitted by controls 126) output steganographically encoded analogsignal A′ to additional appliances such as, for example, a digitalrecorder 102 and/or an analog recorder 514. In this example, each ofappliances 100A, 100B has a rights management component 124 thatelectronically manages rights based on the steganographically encodedcontrols 126. Because the steganographically encoded controls 126persist, recording devices 102, 514 record the steganographicallyencoded controls 126 in any recordings they make of signal A′. In onenon-limiting example, when rights control information is encoded insteganographic sound recordings that are broadcast via radio or someother method, an airplay audit service can sample stations in a givenmarket and identify particular properties being broadcast from “objectidentifier” information contained in the steganographically encoded VDEcontainer.

FIG. 15C shows an example signal distribution arrangement 520 in whichthe steganographically encoded analog signal A′ is initially distributedin the same manner as shown in FIG. 15A, and is then converted by anelectronic appliance 100G such as a personal computer, for example, intoa digital signal D. In this example, appliance 100G includes a rightsmanagement component 124 that manages rights based on steganographicallyencoded controls 126. Appliance 100G may convert received analog signalA′ into digital form for distribution to and processing by digitalappliances such as a digital high definition television 106B, a digitaloptical disk recorder 102, and/or a digital tape recorder 118 a. In oneexample, the steganographically encoded control information 126 persistswithin the digitized signal D. In another example, appliance 100Gremoves the steganographic encoding from received analog signal A′ andoutputs a digital signal D that is “clean” and free of steganographicencoding—but is otherwise protected so that it remains persistentlyassociated with the now-digital control information 126 (which appliance100G may distribute, for example, within secure electronic containers136 and digital, encrypted form. In one specific example, appliance 100Gmay package the received, digitized content from analog signal A′ withinthe same digital electronic container 136 that also contains associatedcontrol information that appliance 100G steganographically decodes fromanalog signal A′. In another specific example, appliance 100G maydistribute controls 126 independently of the digital signal D—but undercircumstances in which the rights management components 124 within eachof digital appliances 106B, 102 and 118A all securely associate thecontrol information with the now-digital content.

FIG. 15D shows a similar distribution arrangement 530 for analog radioor other audio signals. In this example, appliance 100G may include adigital radio receiver that receives analog radio signal A′ and convertsit into a digital information signal for distribution to digitalrecorders 102, 514A. As discussed above, appliance 100G may distributethe digitized analog signal A′ with steganographic encoding toappliances 102, 514A—each of which includes a rights managementcomponent 124 that may recover the steganographically-encoded controlinformation 126 and perform rights management functions based thereon.In another particular example, appliance 100G may remove thesteganographic encoding from the content before distributing it indigital form—and user other techniques (such as those described in theabove-referenced Ginter et al. patent specification) to provide a secureassociation between the now-digital content and the digital controlinformation 126.

FIG. 15E shows yet another example distribution arrangement 540 in whichdigital appliances 102, 100G distribute information in digital form to adigital television 106B. For example, appliance 102 may provide digitalvideo signals D to digital television 106B by playing them back form DVD104. DVD player 102 may provide controls 126 within electronic digitalcontainers 136 to digital television 106B. Digital television 106B mayinclude a rights management component 124C that manages rights in thedigital content based on digitally-provided control information 126.Similarly, computer 100G may receive digital content and associatedcontrol information 126 from a digital network 144, and provide digital,video signals D and associated controls 126 to digital television 106B.

In this example, digital television 106B includes an analog output thatmay provide analog television signals to additional devices, such as,for example, an analog video cassette recorder 118. In this example, therights management component 124C within digital television 106B maysteganographically encode the analog television signal A with controls126 and associated organizational structure(s) 136 before releasing theanalog signal to the outside world.

FIG. 15F shows a further example arrangement 550 in which a digitalappliance 100G such as a personal computer receives digital video signalD and converts it into various analog television signal formats (e.g.,NTSC/PAL and/or RGB) for output to analog devices such as an analog VCR118, an analog set top box 108 and/or an analog television set 106A. Inthis example, a rights management component 124G within digitalappliance 100G steganographically encodes the received digital controls126 onto the analog signal A′, A″ before releasing the analog signal tothe additional appliances 118, 106A, 108.

While the invention has been described in connection with what ispresently considered to be the most practical and preferred embodiment,it is to be understood that the invention is not to be limited to thedisclosed embodiment, but on the contrary, is intended to cover variousmodifications and equivalent arrangements included within the spirit andscope of the appended claims.

1. A rights management method comprising the steps of: (a) receiving aninformation signal at a first device over a communication channel, theinformation signal comprising a content portion that has beensteganographically encoded with control information, the controlinformation being intertwined with the content portion of theinformation signal in locations based on the available bandwidth of thecommunications channel, the control information comprising an indicationof a number of times the content portion may be rendered by a givendevice, as well as an indication of whether at least part of the contentportion may be copied to another device, and whether the controlinformation must be refreshed prior to copying to another device; (b)steganographically decoding the received information signal to recoverthe control information; (c) using the control information to determinewhether the content portion may be rendered by the first device; (d)rendering the content portion on the first device if permitted by thecontrol information; (e) using the control information to determinewhether at least part of the information signal may be copied to asecond device; (f) refreshing the control information using a refreshdecoder and steganographically encoding the content portion with therefreshed control information, if permitted by the control information;and (g) copying at least part of the information signal to the seconddevice if permitted by the control information.
 2. A method as in claim1, in which the control information further comprises an indication thatthe content may be copied only to appliances that are capable ofenforcing the control information.
 3. A method as in claim 1, in whichthe control information further comprises an expiration date after whichthe content cannot be used.
 4. A method as in claim 1, in which thecontrol information further comprises an indication that the content maybe rendered on an audio or video output.
 5. A method as in claim 1, inwhich the control information further comprises an indication of thenumber of times the information signal may be copied.
 6. A method as inclaim 1, further comprising using the control information to govern atleast one rendering of the content on the first device, includingdegrading the quality of the content.
 7. A method as in claim 1, furthercomprising rendering the content on the second device if permitted bythe refreshed control information.
 8. A method as in claim 7, furthercomprising using the refreshed control information to govern at leastone rendering of the content on the second device, including degradingthe quality of the content.